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Description 

METHODS AND APPARATUS FOR 
DISPLAYING APPLICATION OUTPUT ON 
DEVICES HAVING CONSTRAINED SYSTEM 

RESOURCES 

FIELD OF THE INVENTION 

[0001] The present invention relates generally to displaying at 

client devices the output of application programs execut- 
ing on server devices and, more particularly, to techniques 
and apparatus for displaying the output of application 
programs on devices having constrained system re- 
sources. 

BACKGROUND OF THE INVENTION 

[0002] Technologies for providing remote access to networked 
resources include a variety of client/server software com- 
binations. One of these combinations is often referred to 
as a "thin-client" or a "distributed application processing" 
system. In these systems, an application program is exe- 



cuted by a server computing device, usually referred to as 
the "application server," on behalf of one or more client 
computing devices, usually referred to as the "thin-client" 
or the "thin-client application." Only input to the applica- 
tion received from the user at the thin-client and output 
produced by an application executing on the application 
server are transmitted between the thin-client and the ap- 
plication server. Thin-client computing architectures are 
popular implementations for providing remote connectiv- 
ity to applications and other system resources. Examples 
of such systems include: Citrix MetaFrame Presentation 
Server software in combination with Intelligent Computing 
Architecture (ICA) clients, available from Citrix Systems, 
Inc. of Fort Lauderdale, Florida; X servers in combination 
with X Windows clients available from the X Consortium; 
and Microsoft Windows NT Server 4.0 Terminal Server Edi- 
tion in combination with Remote Display Protocol (RDP) 
clients, available from Microsoft Corporation of Redmond, 
Washington. 

[0003] Because a client in a thin-client computing architecture 
does not execute the application program and is required 
to transmit only user input to the application server and 
display only output of the application executing on the 



application server, the client device may offer limited 
amounts of memory, slower communication subsystems, 
and limited system resources without degradation in per- 
formance that is noticeable to the user. A personal com- 
puter, workstation, or other similar computing device typ- 
ically provides ample system resources to execute the 
thin-client application and communicate with the applica- 
tion server. 

[0004] However, more users requiring remote connectivity are 
using computing devices as thin-clients that do not pro- 
vide sufficient memory, network resources, or proper op- 
erating system environments to function as thin-clients, 
such as cell phones and personal digital assistants. For 
example, many current cell phones provide less than 1 
Megabyte of random access memory, which is generally 
not sufficient for execution of the thin-client application. 
Further, it is often useful for an embedded system to ac- 
cess an application server for application output. Typi- 
cally, these systems are also limited in resources such as 
memory. 

[0005] it would be useful to have a system allowing client devices 
having constrained system resources, such as limited 
memory, to interact with application programs executing 



on application servers. 
BRIEF SUMMARY OF THE INVENTION 

[0006] The present invention enables low-end client devices, 

such as cell phones, personal digital assistants, and em- 
bedded systems, to interact with application programs 
executing on application servers, allowing applications to 
accessed remotely from various locations. 

[0007] | n one aspect the present invention relates to a system for 
displaying at a user device output produced by an appli- 
cation program executing on a server. The system in- 
cludes an application server executing an application pro- 
gram. A proxy server receives data from the application 
server, the data representing a screen of graphical display 
output produced by the application program. A user de- 
vice executes a client application that receives static im- 
age data from the proxy server. The received static image 
data represents the screen of graphical display output 
produced by the application program. 

[0008] | n another aspect the present invention relates to a 

method for displaying at a user device output produced 
by an application program executing on a server. An ap- 
plication server executes an application producing a 
screen of graphical user interface data and transmits to a 



proxy server the screen of produced graphical user inter- 
face data. The proxy server transmits to a user device 
static image data representing at least a portion of the 
screen of produced graphical user interface data. The user 
device displays the transmitted static image data. 

[0009] | n s ti|| another aspect the present invention relates to an 
apparatus for displaying at a user device output produced 
by an application program executing on a server. The ap- 
paratus includes a first protocol handler receiving from an 
application server data in a first protocol format, the data 
representative of a screen of graphical display output pro- 
duced by an application executing on the application 
server. The apparatus also includes a second protocol 
handler transmitting to a client application for display 
static image data in a second protocol format, the static 
image data representative of at least a portion of the 
screen of graphical display output received by the first 
protocol handler. 

[0010] | n y e t another aspect the present invention relates to a 
method for displaying at a user device graphical display 
output produced by an application program executing on 
a server. Static image data is received from an application 
server, via a first protocol, representative of a screen of 



graphical display output produced by an application exe- 
cuting on the application server. The static image data is 
transmitted to a client application for display via a second 
protocol, the static image data representative of at least a 
portion of the screen of graphical display output produced 
by the application executing on the application server. 

[° 011 ] In still yet another aspect the present invention relates to 
a system for displaying at a user device output produced 
by an application program executing on a server. The sys- 
tem includes an application server executing an applica- 
tion program. The system also includes a proxy server re- 
ceiving from the application server data that represents a 
screen of graphical display output produced by the appli- 
cation program via a presentation-level protocol. The sys- 
tem further includes a user device executing a client ap- 
plication, the client application receiving from said proxy 
server static image data representing the screen of graph- 
ical display output produced by the application program 
via HyperText Transfer Protocol (HTTP) commands. 

[0012] | n another aspect the present invention relates to a 

method for displaying at a user device output produced 
by an application program executing on a server. An ap- 
plication server executes an application that produces a 



screen of graphical user interface data. The application 
server transmits to a proxy server, via a presentation-level 
protocol, the screen of produced graphical user interface 
data. The proxy server transmits to a user device, via Hy- 
perText Transfer Protocol (HTTP) commands, static image 
data representing at least a portion of the screen of pro- 
duced graphical user interface data. The user device dis- 
plays the transmitted static image data. 

[0013] | n s tj|| another aspect, the present invention relates to an 
article of manufacture having embodied thereon com- 
puter-readable program means for displaying at a user 
device output produced by an application program exe- 
cuting on a server. The article of manufacture includes: 
computer-readable program means for transmitting to a 
proxy server a screen of graphical user interface data pro- 
duced by an application executing on the server; com- 
puter-readable program means for communicating to a 
user device, by the proxy server, static image data repre- 
senting at least a portion of the screen of produced 
graphical user interface data; and computer-readable 
program means for displaying, by the user device, the 
transmitted static image data. 

[0014] | n a s tj|| further aspect the present invention relates to an 



article of manufacture having embodied thereon com- 
puter-readable programs means for displaying at a user 
device graphical display output produced by an applica- 
tion program executing on a server. The article of manu- 
facture includes: computer-readable program means for 
receiving from an application server, via a first protocol, 
data representative of a screen of graphical display output 
produced by an application executing on the application 
server; and computer-readable programs means for 
transmitting to a client application for display, via a sec- 
ond protocol, static image data representative of at least a 
portion of the screen of graphical display output produced 

by the application executing on the application server. 
BRIEF DESCRIPTION OF THE DRAWINGS 

[0015] These and other aspects of this invention will be readily 
apparent from the detailed description below and the ap- 
pended drawings, which are meant to illustrate and not to 
limit the invention, and in which: 

[0016] FIG. 1 is a block diagram of one embodiment of a system 
for providing application output to devices having con- 
strained system resources; 

[0017] piGs. 2A and 2B are block diagrams depicting embodi- 
ments of computers useful in connection with the present 



invention; 

[0018] FIG. 3 is a flowchart depicting one embodiment of the op- 
eration of a system for providing application output to 
devices having constrained system resources; 

[0019] FIG. 4 is a diagrammatic representation of one embodi- 
ment of a protocol used to communicate application out- 
put to devices having constrained systems resources. 
DETAILED DESCRIPTION OF THE INVENTION 

[0020] Referring now to FIG. 1, a system 100 for providing appli- 
cation output to a client device having constrained system 
resources includes an application server 110, a proxy 
server 150, and a client 140. Although only one applica- 
tion server 110, proxy server 150, and client 140 is de- 
picted in the embodiment shown in FIG. 1, it should be 
understood that the system may provide multiple ones of 
any or each of those components. For example, in one 
embodiment, the system 100 includes multiple, logically- 
grouped application servers 110, each of which are avail- 
able to execute applications on behalf of a client 140. In 
these embodiments, the logical group of servers may be 
referred to as a "server farm." In other embodiments, 
multiple proxy servers 150 may be provided. In some of 
these embodiments, the proxy servers may be geographi- 



cally dispersed. 

[0021] The application server 110 executes one or more applica- 
tion programs 122, 124, 126, 128 on behalf of a client 
140. An application program is any program that pro- 
cesses data to provide output and that uses an operating 
system for access to system resources. Exemplary appli- 
cation programs include: word processing applications, 
such as MICROSOFT WORD, manufactured by Microsoft 
Corporation of Redmond, Washington; spreadsheet pro- 
grams, such as MICROSOFT EXCEL, manufactured by Mi- 
crosoft Corporation; electronic mail programs, such as 
MICROSOFT OUTLOOK, manufactured by Microsoft Corpo- 
ration and CROUPWISE, manufactured by Novell Corp. of 
Provo, Utah; and productivity suites such as STAR OFFICE, 
manufactured by Sun Microsystems of Mountain View, 
California. 

[0022] The application server 110 communicates with the proxy 
server 150 over a first network 125. The first network 125 
can be a local area network (LAN), a metropolitan area 
network (MAN), or a wide area network (WAN) such as the 
Internet. The application server 110 and the proxy server 
150 may connect to the first network 125 through a vari- 
ety of connections including standard telephone lines, 



LAN or WAN links (e.g., Tl, T3, 56 kb, X.25), broadband 
connections (ISDN, Frame Relay, ATM), and wireless con- 
nections. Connections between the application server 110 
and the proxy server 150 may use a variety of data-link 
layer communication protocols (e.g., TCP/IP, IPX, SPX, 
NetBIOS, NetBEUI, SMB, Ethernet, ARCNET, Fiber Dis- 
tributed Data Interface (FDDI), RS232, IEEE 802.11, IEEE 
802.11a, IEE 802.11b, IEEE 802. llg and direct asyn- 
chronous connections). 
[0023] The proxy server 150 executes one or more thin-client 
applications 152, 154 such as a Remote Display Protocol 
client, manufactured by Microsoft Corporation or an ICA 
client, manufactured by Citrix Systems, Inc. of Fort Laud- 
erdale, Florida. The application server 110 communicates 
the output of the application programs 122, 124, 126, 
128 to thin-client applications 152, 154 executing on the 
proxy server 150 and receives user input directed to the 
application programs 122, 124, 126, 128 from the thin- 
client application 152, 154. The application server 110 
communicates with the thin-client applications 152, 154 
over network 125 using a presentation-layer protocol 
such as the Independent Computing Architecture (ICA) 
protocol, available from Citrix Systems, Inc. of Fort Laud- 



erdale, Florida or the Remote Display Protocol (RDP), 
available from Microsoft Corporation. Although only two 
thin-client applications are depicted in the embodiment 
shown in FIG, 1, the proxy server 150 may host any num- 
ber of thin-client applications 152, 154. 

[0024] The proxy server 150 also executes a proxy server appli- 
cation 158. The proxy server application 158 may be an 
application program, a subsystem or a service. The proxy 
server application 158 manages the thin-client applica- 
tions 152, 154 hosted by the proxy server 150. The proxy 
server application also transmits application output re- 
ceived by the thin-client applications 152, 154 to the 
client device 140 and transmits user input received from 
the client device 140 to the appropriate thin-client appli- 
cation 152, 154 executing on the proxy server 150. 

[0025] The proxy server application 158 executing on the proxy 
server 150 communicates with the client 140 over a sec- 
ond network 175. For embodiments in which the client 
140 is an embedded system, the client 140 and the proxy 
server 150 may connect to the second network 175 
through a variety of connections including standard tele- 
phone lines, LAN or WAN links (e.g., Tl, T3, 56 kb, X.25), 
broadband connections (ISDN, Frame Relay, ATM), and 



wireless connections. Connections between the client 140 
and the proxy server 150 may use a variety of data-link 
layer communication protocols (e.g., TCP/IP, IPX, SPX, 
NetBIOS, NetBEUI, SMB, Ethernet, ARCNET, Fiber Dis- 
tributed Data Interface (FDDI), RS232, IEEE 802.11, IEEE 
802.11a, IEE 802.11b, IEEE 802. llg and direct asyn- 
chronous connections). 

[0026] | n other embodiments, the client device 140 is a mobile 
device, such as a cellular telephone or a personal digital 
assistant. In these embodiments, the client 140 and the 
proxy server application 158 connect to the second net- 
work using any one of a number of well-known protocols 
from the GSM or CDMA families, such as W-CDMA. These 
protocols support commercial wireless communication 
services and W-CDMA, in particular, is the underlying 
protocol supporting i-Mode and mMode services, offered 
by NTT DoCoMo. 

[0027] The client device 140 executes a client application 146. 
The client application transmits and receives http or https 
requests to and from the proxy server application 158. 
The client application 148 transmits user input directed to 
an executing application program 122, 124, 126, 128 to 
the proxy server application 158 over the second network 



175. It is also responsible for rendering graphical output 
on the screen of the client device corresponding to the 
output of the application program 122, 124, 126, 128 ex- 
ecuting on the application server 110. 

[0028] | n many embodiments, the application server 110 and the 
proxy server 150 are provided as personal computer or 
computer servers, of the sort manufactured by the 
Hewlett-Packard Corporation of Palo Alto, California or 
the Dell Corporation of Round Rock, TX. For embodiments 
in which the client device 140 is an embedded system, the 
client device 140 may also be provided as a personal 
computer. FIGs. 2A and 2B depict block diagrams of a 
typical computer 200 useful as the application server 110, 
the proxy server 150, or the client device in those embod- 
iments. As shown in FIGs. 2A and 2B, each computer 200 
includes a central processing unit 202, and a main mem- 
ory unit 204. Each computer 200 may also include other 
optional elements, such as one or more input/output de- 
vices 230a-230n (generally referred to using reference 
numeral 230), and a cache memory 240 in communication 
with the central processing unit 202. 

[0029] The central processing unit 202 is any logic circuitry that 
responds to and processes instructions fetched from the 



main memory unit 204. In many embodiments, the central 
processing unit is provided by a microprocessor unit, such 
as: the 8088, the 80286, the 80386, the 80486, the Pen- 
tium, Pentium Pro, the Pentium II, the Celeron, or the Xeon 
processor, all of which are manufactured by Intel Corpo- 
ration of Mountain View, California; the 68000, the 
68010, the 68020, the 68030, the 68040, the PowerPC 
601, the PowerPC604, the PowerPC604e, the MPC603e, 
the MPC603ei, the MPC603ev, the MPC603r, the 
MPC603p, the MPC740, the MPC745, the MPC750, the 
MPC755, the MPC7400, the MPC7410, the MPC7441, the 
MPC7445, the MPC7447, the MPC7450, the MPC7451, the 
MPC7455, the MPC7457 processor, all of which are manu- 
factured by Motorola Corporation of Schaumburg, Illinois; 
the Crusoe TM5800, the Crusoe TM5600, the Crusoe 
TM5500, the Crusoe TM5400, the Efficeon TM8600, the 
Efficeon TM8300, or the Efficeon TM8620 processor, 
manufactured by Transmeta Corporation of Santa Clara, 
California; the RS/6000 processor, the RS64, the RS 64 II, 
the P2SC, the POWER3, the RS64 III, the POWER3-II, the RS 
64 IV, the POWER4, the POWER4+, the POWERS, or the 
POWER6 processor, all of which are manufactured by In- 
ternational Business Machines of White Plains, New York; 



or the AMD Opteron, the AMD Athalon 64 FX, the AMD 
Athalon, or the AMD Duron processor, manufactured by 
Advanced Micro Devices of Sunnyvale, California. 
[0030] Main memory unit 204 may be one or more memory chips 
capable of storing data and allowing any storage location 
to be directly accessed by the microprocessor 202, such 
as Static random access memory (SRAM), Burst SRAM or 
SynchBurst SRAM (BSRAM), Dynamic random access mem- 
ory (DRAM), Fast Page Mode DRAM (FPM DRAM), Enhanced 
DRAM (EDRAM), Extended Data Output RAM (EDO RAM), 
Extended Data Output DRAM (EDO DRAM), Burst Extended 
Data Output DRAM (BEDO DRAM), Enhanced DRAM 
(EDRAM), synchronous DRAM (SDRAM), JEDEC SRAM, 
PC100 SDRAM, Double Data Rate SDRAM (DDR SDRAM), 
Enhanced SDRAM (ESDRAM), SyncLink DRAM (SLDRAM), 
Direct Rambus DRAM (DRDRAM), or Ferroelectric RAM 
(FRAM). 

[0031] | n the embodiment shown in FIG. 2A, the processor 202 
communicates with main memory 204 via a system bus 
220 (described in more detail below). FIG. 2B depicts an 
embodiment of a computer system 200 in which the pro- 
cessor communicates directly with main memory 204 via a 
memory port. For example, in FIG. 2B the main memory 



204 may be DRDRAM. 

[0032] piGs. 2A and 2B depict embodiments in which the main 

processor 202 communicates directly with cache memory 
240 via a secondary bus, sometimes referred to as a 
"backside" bus. In other embodiments, the main processor 
202 communicates with cache memory 240 using the sys- 
tem bus 220. Cache memory 240 typically has a faster re- 
sponse time than main memory 204 and is typically pro- 
vided by SRAM, BSRAM, or EDRAM. 

[0033] | n the embodiment shown in FIG. 2A, the processor 202 
communicates with various I/O devices 230 via a local 
system bus 220. Various busses may be used to connect 
the central processing unit 202 to the I/O devices 230, in- 
cluding a VESA VL bus, an ISA bus, an EISA bus, a Mi- 
croChannel Architecture (MCA) bus, a PCI bus, a PCI-X 
bus, a PCI-Express bus, or a NuBus. For embodiments in 
which the I/O device is an video display, the processor 
202 may use an Advanced Graphics Port (AGP) to commu- 
nicate with the display. FIG. 2B depicts an embodiment of 
a computer system 200 in which the main processor 202 
communicates directly with I/O device 230b via Hyper- 
Transport, Rapid I/O, or InfiniBand. FIG. 2B also depicts an 
embodiment in which local busses and direct communica- 



tion are mixed: the processor 202 communicates with I/O 
device 230a using a local interconnect bus while commu- 
nicating with I/O device 230b directly. 

[0034] a wide variety of I/O devices 230 may be present in the 
computer system 200. Input devices include keyboards, 
mice, trackpads, trackballs, microphones, and drawing 
tablets. Output devices include video displays, speakers, 
inkjet printers, laser printers, and dye-sublimation print- 
ers. An I/O device may also provide mass storage for the 
computer system 200 such as a hard disk drive, a floppy 
disk drive for receiving floppy disks such as 3.5-inch, 
5.25-inch disks or ZIP disks, a CD-ROM drive, a CD-R/RW 
drive, a DVD-ROM drive, tape drives of various formats, 
and USB storage devices such as the USB Flash Drive line 
of devices manufactured byTwintech Industry, Inc. of Los 
Alamitos, California. 

[0035] | n further embodiments, an I/O device 230 may be a 

bridge between the system bus 220 and an external com- 
munication bus, such as a USB bus, an Apple Desktop Bus, 
an RS-232 serial connection, a SCSI bus, a FireWire bus, a 
FireWire 800 bus, an Ethernet bus, an AppleTalk bus, a 
Gigabit Ethernet bus, an Asynchronous Transfer Mode 
bus, a HIPPI bus, a Super HIPPI bus, a SerialPlus bus, a 



SCI/LAMP bus, a FibreChannel bus, or a Serial Attached 
small computer system interface bus. 
[0036] General-purpose desktop computers of the sort depicted 
in FIGs. 2A and 2B typically operate under the control of 
operating systems, which control scheduling of tasks and 
access to system resources. Typical operating systems in- 
clude: MICROSOFT WINDOWS, manufactured by Microsoft 
Corp. of Redmond, Washington; MacOS, manufactured by 
Apple Computer of Cupertino, California; OS/2, manufac- 
tured by International Business Machines of Armonk, New 
York; and Linux, a freely-available operating system dis- 
tributed by Caldera Corp. of Salt Lake City, Utah, among 
others. 

[0037] For embodiments in which the client device 140 is a mo- 
bile device, the client device may be a JAVA-enabled cel- 
lular telephone, such as the i50sx, i55sr, i58sr, i85s, i88s, 
i90c, i95cl, or the imllOOO, all of which are manufactured 
by Motorola Corp. of Schaumburg, Illinois, the 6035 or the 
7135, manufactured by Kyocera of Kyoto, Japan, or the 
i300 or i330, manufactured by Samsung Electronics Co., 
Ltd., of Seoul, Korea. In other embodiments in which the 
client device 140 is mobile, it may be a personal digital 
assistant (PDA) operating under control of the PalmOS op- 



erating system, such as the Tungsten W, the VII, the Vllx, 
the i705, all of which are manufactured by palmOne, Inc. 
of Milpitas, California. In further embodiments, the client 
device 140 may be a personal digital assistant (PDA) oper- 
ating under control of the PocketPC operating system, 
such as the IPAQ4155, IPAQ 5555, iPAQ 1945, iPAQ 
2215, and iPAQ4255, all of which manufactured by 
Hewlett-Packard Corporation of Palo Alto, California, the 
ViewSonic V36, manufactured by ViewSonic of Walnut, 
California, or the Toshiba PocketPC e405, manufactured 
by Toshiba America, Inc. of New York, New York. In still 
other embodiments the client device is a combination 
PDA/telephone device such as the Treo 180, Treo 270 or 
Treo 600, all of which are manufactured by palmOne, Inc. 
of Milpitas, California. In still further embodiment, the 
client device 140 is a cellular telephone that operates un- 
der control of the PocketPC operating system, such as the 
MPx200, manufactured by Motorola Corp. 
[0038] FIG. 3 depicts the operation of the system just described 
in Figs. 1-2B. The client application 146 transmits to the 
proxy server application 158 a request to have an applica- 
tion program 122, 124, 126, 128 executed on its behalf 
(step 302). In some embodiments, the http request trans- 



mitted by the client application 146 includes the name of 
the application the user wants to have executed. In other 
embodiments, the request may identify a file on which the 
user wants to work. In these embodiments, the request 
includes the file type and the proxy server application 158 
identifies one or more application programs capable of 
processing the file. For example, the proxy server applica- 
tion 158 may look up the application program to use from 
a table mapping file types to specific application pro- 
grams associated with those files types. In still further 
embodiments, the http request transmitted by the client 
application 146 may specifically identify a particular pub- 
lished desktop or particular server 100 on which the ap- 
plication program 122, 124, 126, 128 should be exe- 
cuted. 

[0039] The proxy server application 158 spawns a thin client ap- 
plication 152, 154 (step 322) and provides the thin-client 
application 152, 154 with the identity of the requested 
program. The thin-client application 152, 154 operates in 
a manner well-known in the art, except that the thin- 
client application 152, 154 sets aside a block of memory 
to which the thin-client writes application output (step 
342). Put another way, the thin-client application 152, 



154 writes application output to a virtual screen main- 
tained in memory rather than to a visual display, as is 
usually done in the art. In some embodiments, however, 
the thin-client application 152, 154 writes to both a vir- 
tual screen and to a traditional display screen. 

[0040] The thin-client application 152, 154 transmits a request 
to a server 110 to begin execution of an application pro- 
gram 122, 124, 126, 128 (step 344). In some embodi- 
ments the thin-client application 152, 154 transmits the 
request to one server in a server farm, which responds to 
the thin-client application 152, 154 with the address of a 
server 110 hosting the requested application program 
122, 124, 126, 128. In other embodiments, the thin-client 
application 152, 154 transmits the request to a service 
access point, which returns a document to the client de- 
vice 140 that provides the client application 146 with the 
necessary information to connect to the appropriate ap- 
plication server 110. In all of these embodiments, the 
thin-client application 152, 154 initiates a connection 
with the identified server 110. 

[0041] | n other embodiments, the thin-client application 152, 

154 sets up a virtual screen memory area (step 342) after 
transmitting the application request to the application 



server 110 (step 344). In still other embodiments, these 
actions happen substantially simultaneously. 
[0042] The server 110 begins executing the requested applica- 
tion program 122, 124, 126, 128 (step 382) and transmits 
graphical application output over the first network 125 
using a thin-client presentation protocol (step 384). Vari- 
ous embodiments of the steps taken by the application 
server 110 to form presentation-level protocol packets for 
transmission to the thin-client application 152, 154 are 
described in United States Patent Nos. 6,141,737, 
6,118,899, 6,081,623, 6,057,857 and 6,016,535, the en- 
tire contents of which are incorporated herein by refer- 
ence. 

[0043] jhe thin-client application 152, 154 decodes presentation 
protocol packets received from the application server 110 
and writes graphical application output represented by the 
presentation protocol packets to the virtual screen mem- 
ory (step 346). In many embodiments, the virtual screen 
memory is a screen buffer, i.e., the virtual screen memory 
area stores a bitmap representation of data required to 
form a visual display of the graphical output. 

[0044] | n the embodiment shown in FIG. 3, the client application 
146 transmits a request for a static image representing 



the current state of the graphical application output to the 
proxy server application (step 304). The client application 
may request the current state of the graphical application 
output every minute, every thirty seconds, every 15 sec- 
onds, every 10 seconds, every 5 seconds, every other sec- 
ond, once a second, twice a second, five times a second, 
ten times a second, twenty times a second, or thirty times 
a second. In other embodiments, the proxy server appli- 
cation 158 periodically transmits a static image repre- 
senting the current state of the graphical application out- 
put to the client application 146. In these other embodi- 
ments, the proxy server application 158 may transmit up- 
dates to the client application 146 every minute, every 
thirty seconds, every 15 seconds, every 10 seconds, every 
5 seconds, every other second, once a second, twice a 
second, five times a second, ten times a second, twenty 
times a second, or thirty times a second. Alternatively, the 
proxy server application 158 may transmit an update to 
the client application 146 when the proxy server applica- 
tion 158 determines that a predetermined percentage of 
the virtual screen memory area has changed state or 
whenever a request for the current state of the graphical 
application output is received by the proxy server applica- 



tion. 

[0045] Referring back to FIG. 3, the proxy server application 158 
receives the transmitted request (step 324) and creates a 
static image file using the virtual screen memory of the 
appropriate thin-client application 152, 154 as input (step 
326). In some embodiments, the client application 146 
stores the state of its connection with the proxy server 
application 158, for example, in a "cookie." In these em- 
bodiments, the transmitted request (step 324) may in- 
clude a session identifier that the proxy server application 
158 uses to select from which virtual screen memory to 
create the static image file. In further of these embodi- 
ments, the request includes a unique identifier associated 
with the client device 140. For example, for embodiments 
in which the client device 140 is a cell phone, the unique 
identifier may be the telephone number associated with 
the cell phone. 

[0046] | n other embodiments, the client application 146 does not 
store the state of its connection with the proxy server ap- 
plication 158. In these embodiments, the session must be 
identified in the transmitted request (step 324) in a man- 
ner that does not reveal the session identifier to an eaves- 
dropper. For example, the portion of the transmitted http 



request (step 324) that identifies the session may be en- 
crypted using a private key shared between the client ap- 
plication 146 and the proxy server application 158. In fur- 
ther of these embodiments, the request includes a unique 
identifier associated with the client device 140, which may 
also be encrypted to protect it from an eavesdropper. For 
example, for embodiments in which the client device 140 
is a cell phone, the unique identifier may be the telephone 
number associated with the cell phone. 

[0047] The static image created by the proxy server application 
158 may be in any one of a number of standard formats, 
including JPEG, GIF, PNG, TIFF, or BMP. In some embodi- 
ments, the proxy server application 158 calls a Common 
Object Model (COM) Application Programming Interface 
(API) exposed by the thin-client application 152, 154 to 
create the static image. 

[0048] | n some embodiments, the proxy server application 158 
optimizes the size of the static image created. In one of 
these embodiments, the proxy server application 158 de- 
termines the size of the screen image by selecting the 
screen area surrounding the user's current position. The 
user's current position may determined by using the x co- 
ordinate and y coordinate of the last mouse click or the x 



coordinate and y coordinate of the last mouse position to 
be transmitted to the proxy server application 158. The 
amount of the screen above, below, to the left of and to 
the right of the user's last position that is used to create 
the screen image may be set to a predetermined number 
of pixels, or it may be selected based on the client device 
140. 

[0049] | n another embodiment, the proxy server application 158 
increases the amount of image loss that is acceptable. 
That is, the proxy server application 158 uses a lossy 
compression algorithm, such as JPEG, to encode the 
screen image before transmission. The proxy server ap- 
plication 158 may select the amount of image loss that is 
acceptable based on the image type to be transmitted, the 
relative size differential between the screen of the client 
device 158 and the virtual screen memory, the absolute 
size of the screen of the client device 140, the bit depth of 
the screen of the client device 140, or the bandwidth 
available for use over the network 175. 

[0050] | n s ti|| other embodiments, the proxy server application 
158 does not use a static image to transmit screens that 
have no graphical elements. In these embodiments, the 
client application 146 exposes a separate set of program- 



ming interfaces for text drawings commands and for 
graphical commands. For screens having no graphical ele- 
ments, only text drawing calls would be made to render 
the application output screen. 
[0051] | n s ti|| other embodiments, the proxy server application 
158 scales the application output read from the virtual 
screen buffer so that the screen, or a larger portion of the 
screen, is viewable on the screen of the client device 140. 
Alternatively, the scaling may be provided by a COM inter- 
face. 

[0052] The proxy server application 158 transmits the created 

static image file to the client application 146 (step 328). In 
some embodiments, a single session identifier may be 
shared between multiple client devices 140, allowing one 
or more client devices 140 to shadow another client de- 
vice 140, i.e. the output displayed by the "shadowing" 
client devices 140 is the same as the "shadowed" client 
device 140. In one embodiment, the static image file is 
transmitted to client application 146 using HyperText 
Transfer Protocol (http) or Secure HyperText Transfer Pro- 
tocol (https). The protocol used to transmit static image 
files to the client application 146 and to receive client in- 
put from the client application 146 is described in more 



detail in connection with FIG. 4 below. The client applica- 
tion 146 displays the received static image file (step 308). 

[0053] Referring now to FIG. 4, one embodiment of a protocol 
used by the client application 146 and the proxy server 
application 158 to exchange user input and application 
execution output is depicted diagrammatically. In the em- 
bodiment shown in FIG. 4, four protocol commands are 
depicted: "Get Image" 410; "Send String" 420; "Send 
Keystroke" 430; and "Send Mouse Event" 440. 

[0054] a s shown in FIG. 4, the client application 146 transmits to 
the proxy server application 158 an http request 412 
identifying the server to which the request is directed and 
parameters concerning the static image requested by the 
client application 146. In the embodiment shown in FIG. 4, 
the parameters included in the http request transmitted 
by the client application 146 include a starting x coordi- 
nate, a starting y coordinate, an ending x coordinate, an 
ending y coordinate, and a preferred static image file 
type. The proxy server application 158 responds with an 
http packet 414 transmitting the requested static image 
file. 

[0055] The protocol embodiment shown in FIG. 4 also includes 
three input commands, "Send String" 420, "Send 



Keystroke" 430, and "Send Mouse Event" 440. For each of 
these commands, the client application 146 transmits to 
the proxy server application 158 an http packet identify- 
ing the server to which the user input is directed and the 
input. In the case of the "Send String" command 422, the 
user input is a series of alphanumeric characters. In the 
case of the "Send Keystroke" command 432, the user in- 
put is the keystroke. In the case of the "Send Mouse Event" 
command 442, the user input is the x coordinate of the 
mouse event, the y coordinate of the mouse event, and 
whether the mouse event includes a "click." In some em- 
bodiments, the "Send Mouse Event" also include an indi- 
cation of which mouse button was clicked. In each of 
these cases, the proxy server application 158 responds to 
the client application 146 with an "OK" message 424, 434, 
444. 

[0056] Upon receipt of user input 422, 432, 442, the proxy 

server application 158 forwards the user input to the thin- 
client application 152, 154. The thin-client application 
152, 154 forwards the received user input to the applica- 
tion server 110. the application server 110 receives the 
user input and provides it to the application program 122, 
124, 126, 128. 



[0057] The present invention may be provided as one or more 
computer-readable programs embodied on or in one or 
more articles of manufacture. The article of manufacture 
may be a floppy disk, a hard disk, a compact disc, a digi- 
tal versatile disc, a flash memory card, a PROM, a RAM, a 
ROM, or a magnetic tape. In general, the computer-read- 
able programs may be implemented in any programming 
language. Some examples of languages that can be used 
include C, C+ + , C#, or JAVA. The software programs may 
be stored on or in one or more articles of manufacture as 
object code. 

[0058] while the invention has been shown and described with 
reference to specific preferred embodiments, it should be 
understood by those skilled in the art that various 
changes in form and detail may be made therein without 
departing from the spirit and scope of the invention as 
defined by the following claims. 



